Dexterous telemanipulator system

ABSTRACT

A robotic manipulator has a body and a stereoscopic video system movably coupled to the body. The stereoscopic vision system produces a stereoscopic video of an environment of the robotic manipulator. The robotic manipulator also includes two independently remotely controlled arms coupled to opposite sides of the body. Each arm moves in proprioceptive alignment with the stereoscopic video produced by the stereoscopic video system in response to commands received from a remote control station based on movements performed by an operator at the remote control station.

RELATED APPLICATIONS

This application claims priority to and the benefit of U.S. provisionalapplication No. 61/466,902, filed on Mar. 23, 2011, titled “A MobileRobotic Manipulator System”, and U.S. provisional application No.61/466,904, filed on Mar. 23, 2011, titled “Dexterous TelemanipulatorSystem,” the entireties of which applications are incorporated byreference herein.

GOVERNMENT RIGHTS IN THE INVENTION

This invention was made with government support under Contract No.HSHQDC-10-C-00118 awarded by the Department of Homeland Security. Thegovernment has certain rights in this invention.

FIELD OF THE INVENTION

The invention relates generally to teleoperated robotic manipulatorsystems.

BACKGROUND

Teleoperated systems generally include a man-machine interface throughwhich a human operator can remotely control a robotic manipulator. Anexample of a teleoperated system is the Da Vinci Surgical System, madeby Intuitive Surgical of Sunnyvale, Calif. The Da Vinci Surgical Systemhas three robotic arms that operate as tools and a fourth arm thatcarries a two-lens camera. A surgeon sits at a console and, whilelooking at the stereoscopic image captured by the robotic arm with thecamera, exercises hand controls and/or foot controls that move the otherarms. The movements of the controls translate into micro-movements ofthe instruments. Surgeons have used the Da Vinci Surgical System toperform minimally invasive surgery remotely.

SUMMARY

In one aspect, the invention relates to a robotic manipulator comprisinga body and a stereoscopic video system movably coupled to the body. Thestereoscopic vision system produces a stereoscopic video of anenvironment of the robotic manipulator. The robotic manipulator furthercomprises two independently remotely controlled arms coupled to oppositesides of the body. Each arm moves in proprioceptive alignment with thestereoscopic video produced by the stereoscopic video system in responseto commands received from a remote control station based on movementsperformed by an operator at the remote control station.

In another aspect, the relates to a system comprising a roboticmanipulator with a servo actuator subsystem having two independentlyremotely controlled arms coupled to opposite sides of a body and avideo-capture subsystem that produces a stereoscopic video of a localenvironment of the robotic manipulator. The robotic manipulator furthercomprises a computational host subsystem that transmits movementcommands to the servo actuator subsystem. The commands cause the servoactuator subsystem to move each arm in proprioceptive alignment with thestereoscopic video produced by the video-capture subsystem.

In yet another aspect, the invention relates to a method of remotelyoperating a robotic manipulator having a body, a stereoscopic videosystem movably coupled to the body, and two remotely controlled armscoupled to opposite sides of the body. The method comprises capturing,by the video system, a stereoscopic video of an environment of therobotic manipulator, and independently moving, in response to commandsreceived from a remote control station, each arm in proprioceptivealignment with the stereoscopic video produced by the stereoscopic videosystem.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further advantages of this invention may be betterunderstood by referring to the following description in conjunction withthe accompanying drawings, in which like numerals indicate likestructural elements and features in various figures. The drawings arenot necessarily to scale, emphasis instead being placed uponillustrating the principles of the invention.

FIG. 1 is a photograph of an embodiment of a robotic manipulator with astereoscopic video system and dexterous independently remotelycontrolled arms.

FIG. 2 is a functional block diagram of an embodiment of a teleoperatedrobotic manipulator system.

FIG. 3 is a diagram of an embodiment of the robotic manipulator withoutprotective covering.

FIG. 4 is a front view of the robotic manipulator illustrating a widthand height to which the robotic manipulator can fold.

FIG. 5 is a top view of the robotic manipulator illustrating its foldedwidth and depth.

FIG. 6 is a side view of the robotic manipulator illustrating its foldedheight.

FIG. 7 is a diagram of an embodiment of a torso of the roboticmanipulator.

FIG. 8 is a diagram of an embodiment of a shoulder of the roboticmanipulator.

FIG. 9 is a diagram of an embodiment of an upper arm of the roboticmanipulator.

FIG. 10 is a diagram of an embodiment of an elbow of the roboticmanipulator.

FIG. 11 is a diagram of an embodiment of a forearm of the roboticmanipulator

FIG. 12 is a diagram of an embodiment of a wrist of the roboticmanipulator.

FIG. 13 is a diagram of an embodiment of the stereoscopic video systemof the robotic manipulator.

DETAILED DESCRIPTION

Applicants recognized that various telemanipulator principles embodiedin the Da Vinci Surgical System can be extended to non-medical uses, forexample, executing operations within environments hazardous to humanswhile the operator remains at a remote, safe distance. For example, therobotic manipulator systems described herein can be vehicle-mounted andused to approach, deactivate, and render harmless improvised explosivedevices (IED) by bomb squad personnel. Other applications include, butare not limited to, the repair of orbital spacecraft or the capture orstabilizing of orbital objects.

The robotic manipulator systems described herein include a remotelyoperated robotic manipulator with three-dimensional stereoscopic visionand two independently controllable dexterous hands having multipledegrees of freedom (DOF). The arms of the robot manipulator carry toolsor instruments, such as graspers, clamps, cutters, electrical probes, orsensors. The robotic manipulator can serve as an attachment to acoarse-movement vehicle and arm, or operate as a stand-alone unit. Itsdesign is scalable and produces a small form factor that permits therobotic manipulator to enter and operate in tight spaces.

Control of the robotic manipulator, in general, occurs from a remotecontrol station over an RF (radio frequency) link, fiber optic orelectrical cable tether. An operator works at the control station withan intuitive human-machine interface through which to perform complextasks remotely. The control station achieves proprioceptive alignment ofstereoscopic imagery with hand controls; that is, the movement of therobotic arms and hands closely corresponds to the movement of theoperator's arms and hands. Intuitive telemanipulation also includesmotion and rotation clutching, linked camera zoom and movement scalingand transforming movement commands into the camera reference frame fornatural, reliable, and predictable operation.

FIG. 1 shows a photograph of an embodiment of a robotic manipulator 10with a stereoscopic video system 12 and two independently controlleddexterous arms 14-1, 14-2 (generally, 14) movably coupled to a main bodyor torso having a cylinder-shaped forward portion 16A and a back portion16B (generally, torso 16). The torso 16 can mount to a mobile EOD(Explosive Ordnance Disposal) platform, such as Andros F6-A. Thestereoscopic video system 12 is movably coupled to the forward portion16A of the torso. Preferably, the stereoscopic video system 12 hasremote-controlled tilt-zoom-focus capabilities. Embodiments ofstereoscopic video system 12 can be standard definition orhigh-definition (HD).

Each arm 14-1, 14-2 is connected to the cylinder-shaped forward portion16A of the torso by a respective shoulder 18-1, 18-2 (generally, 18).The shoulders 18 are rotatably connected to opposite sides of theforward portion 16A of the torso. Each arm 14 comprises an upper arm 20rotatably connected to the shoulder 18, a forearm 22 rotatably connectedto the upper arm 20 by an elbow 24, and a wrist 26 rotatably connectedto the forearm 22. The wrists 26 can be configured with a variety ofchangeable tools 28, including, for example, a multi-functionalcutter-grasper. The long, slender forearm-to-wrist lessens obstructingthe view of the video system 12 to the work site and tool 28. In oneembodiment, with both arms engaged, the robotic manipulator 10 iscapable of holding approximately a 5-pound payload.

The upper arms 20 can rotate about any of three axes, as described inmore detail in connection with FIG. 8. These three axes intersect at apoint where the shoulder 18 joins the upper arm 20 (i.e., the shoulderjoint). The forearm rotates within the elbow, and the wrist rotatesabout three perpendicular axes, two of which intersect. The ability torotate about any of the seven axes corresponds to six degrees of freedom(DOF) of movement for each arm 14: moving forward and backward, movingleft and right, moving up and down, tilting forward and backward(pitch), tilting side-to-side (roll), and rotating left and right (yaw).The seventh axis of rotation provides redundancy, and the ability ofselecting between many arm configurations when the tool 28 is maintainedin a desired position and orientation.

The static design of the robotic manipulator 10 is to have the center ofmass of the whole arm 14 at the intersection of the three axes ofrotation. Advantageously, this intersection at the shoulder jointensures the center of mass remains at the shoulder joint throughout therange of 3-dimensional motion of the arm 14. This static balance designpromotes stability of the robotic manipulator, enables sensing of thetool and payload forces and not the arm weight, and facilitatesefficient dedication of electrical power to the handling of the tool andpayload.

FIG. 2 shows a functional block diagram of a teleoperated roboticmanipulator system 50 including a control station 52 in communicationwith the robotic manipulator 10 over a communication link 54. Thecommunication link 54 can be, for example, a fiber optic cable (10BaseT), Ethernet cable, or a RF communication link. The robotic manipulator10 also includes a servo actuator subsystem 56, a computational hostsubsystem 58, a video-capture subsystem 60, and a power conversion anddistribution subsystem 62.

Through the control station 52, the operator can reposition the roboticmanipulator 10, individually control the robotic arms 14 simultaneously,and execute camera operations, such as tilt, zoom, motion scaling,tremor filtering, and stereo convergence. Dynamic adjustment to theconvergence angle provides comfortable stereoscopic viewing through awide range of operating depths. In brief overview, the controls station52 comprises computational elements (e.g., processor, memory, software,network interface) for translating interface device movements intocommand data packets, transmitting the commands to the roboticmanipulator, performing hardware-accelerated decompression of videoarriving from the robotic manipulator, presenting graphical interfaceoverlays and haptic displays for notifying the operator of the limits ofthe workspace and of any impending collisions with remote structures.

In brief overview, the control station 52 provides operator feedback inthe form of haptic forces on three axes, tactile vibration on threeaxes, a sense of tool-grip effort, and a three-dimensional binocularview. The control station 52 includes a stereoscopic display 64(preferably capable of high-definition) and haptic-interface devices 66.The display can be implemented with a pair of 15-inch diagonal LCDdisplays with a 1024×800 resolution, for example. Alternatively, thedisplay 64 can be a SXGA resolution (1280×960) Organic LED micro-displayproduced by eMagin Corporation of Bellevue Wash. Other displayalternatives include LCD panels with 120 Hz refresh rate and active LCDshuttered glasses, or 3D displays using filters and passive glasses.

The haptic-interface devices 66 can be implemented with Omega7 desktophaptic interface devices (manufactured by Force Dimension of Nylon,Switzerland), or with a bimanual interface device produced by MimicTechnologies of Seattle, Wash. The Mimic desktop device uses tensionedcables connecting the control handles to measure position and to exertforces to the operator. Servomotors attach to spools that control thetension in the cables. The dual operator hand-control haptic-interfacedevices 66 provide six-DOF movement (left, right, back, forward, up, anddown) and a proportional gripper control. Vibrotactile feedback based on3-axis accelerometers embedded in each manipulator gripper (FIG. 12) isreproduced as resolved forces in the haptic-interface devices 66.Low-pass filtering of the command signals originating from thehaptic-interface devices 66 prevents destabilization in the presence ofarbitrary latencies.

As an alternative to the haptic-interface devices described, in a lowcost implementation, a pair of 6 axis joystick devices (e.g.,SpaceNavigator™ by 3DConnexion of Boston, Mass.) may be used to providemovement commands to the control station. These devices presently do notsupport the display of haptic forces.

In addition, use of low-latency hardware accelerated video CODECs, forlong distance operation using satellite communication techniques, andthe collocation of telepresence operations with the communicationsfacilities helps minimize communication latency. To compensate for longround-trip latencies (e.g., greater than 750 ms), the control station 52can present a display overlay that shows the commanded configuration ofthe remote arm with minimal delay in order to aid the operator incorrectly placing the tools of the robotic manipulator at a desiredtarget without overshoot or hesitation. To the operator, the roboticmanipulator appears to move in immediate real-time response to themovement of the haptic-interface devices. In addition, deliberatelydelaying the vibrational haptic feedback in order to synchronize thefeedback with the decoded video can help avoid inconsistent sensoryqueues that convey a sense of telepresence.

The servo actuator subsystem 56 comprises the various servomotors,encoders, closed-loop position controllers, power amplifiers, andnetwork communication elements of the robotic manipulator 10.

The computational host subsystem 58 translates commands received overthe communications link 54 into low-level instructions for the servoactuator subsystem 56, translates and sends feedback signals receivedfrom the servomotors to the control station 52, and collects,compresses, and transmits video data from the cameras of thevideo-capture subsystem 60 to the control station 52 for display.

The computational host subsystem 58 includes a processor 74 thatexecutes communications and embedded software 76 (stored in memory 78)to direct the functionality of the robotic manipulator 10, such as theability to deploy and stow autonomously, and to calculate rapidly themovement required of each manipulator joint to achieve the motionrequired by a received command from the control station 52. The software76 translates these required movements into instructions expected by thevarious components of the servo actuator subsystem 56 and video-capturesubsystem 60. The computational host subsystem 58 communicates theseinstructions to each subsystem 56, 60 through standard communicationschannels 80, such as USB 2.0, CANBus, or EtherCAT. As an example,commands can be sent to the servo actuator subsystem 56 at 20-500positions per second.

In the absence of position updates from the operator, individual jointsof the arms 14 hold their last commanded positions. The various levelsof counterbalancing designed into the shapes and locations of the “bodyparts” of the robotic manipulator and the program algorithms employed tomaintain the balance throughout their ranges of motion, as described inmore detail below, enable the robotic manipulator to hold thesepositions using minimum electrical power. An objective of the embeddedsoftware 76 is for the arms to remain substantially balanced through alarge workspace, which, for example, can measure 42 inches in width by25 inches in depth by 33 inches in height.

The software 76 also collects feedback information from the servoactuator and video capture subsystems 56, 60, translates and encodes thefeedback information, and transmits the encoded data to the controlstation 52 over the communication link 54. The embedded software 76 canalso detect and handle fault conditions, allowing recovery wheneverpossible.

The video-capture subsystem 60 acquires stereoscopic images of theremote environments, and responds to commands from the computationalhost subsystem 58 to adjust zoom, focus, or camera convergence. In oneembodiment, the video-capture subsystem 60 supports VGA (640×480)stereoscopic video. Other capabilities can include higher displayresolution and remote adjustment of zoom, focus, convergence, and stereobaseline.

The power conversion and distribution subsystem 62 can draw power froman external power source 68, here, for example, shown to be part of amobile host platform 70 to which the robotic manipulator 10 is mounted,and transforms this power into the electrical voltage and current levelsrequired by the various dependent electronic subsystems 56, 58, and 60.

FIG. 3 shows a view of the robotic manipulator 10 without protectivecovering. The stereoscopic video system 12 has an undercarriage 100 withfour rollers 102. These rollers 102 ride on two arcuate rails 104 of thecylinder-shaped forward portion 16A of the torso. A chain drive 106passes through the undercarriage 100 of the video system 12 and anopening 108 in the forward portion 16A, coupling the video system 12 tothe forward portion 16A of the torso. The chain drive 106 operates topull the video system 12 backward or forward along the rails 104, which,because of the cylinder-shape of the forward portion 16A of the torso,causes the video system 12 to tilt up or down, respectively.

In general, the robotic manipulator 10 employs distributed controlarchitecture with independent closed-loop position servos for eachjoint. Specifically, each upper arm 20 includes a servomotor 110 with aspeed reducing gearbox and pulley. A belt 112 couples the pulley of themotor 110 with a pulley 202 (FIG. 10) of the elbow 24. Behind eachshoulder 18 is another motor 114 coupled to another pulley 172 (FIG. 8)of the shoulder 18 by a belt 116. The motors 110, 114 and belts 112, 116can be commercial-off-the-shelf (COTS) components.

For each arm, the back portion 16B of the torso has a motor 120 coupledby a belt 122 to a pulley 164 (FIG. 8) of the shoulder 18 (only themotor 120 and belt 122 for the right arm 14-1 are visible in FIG. 3).Each belt 112, 116, and 122 is flat with integral teeth for meshing withteeth of the pulleys. In addition, the separate belts provide shockisolation. Further, a low gear ratio mechanical transmission providesmanipulator backdrive-ability and low reflected inertia, mitigating therisk of damage to the robotic manipulator or to structures in the eventof their collision.

The mass of the motor 110 is disposed on the opposite side of theshoulder 18 from the remainder of the arm (i.e., elbow, forearm, andwrist) to provide a counterweight that counterbalances the arm 14approximately at the point of 3-axis-intersection within the shoulderjoint. Advantageously, counterbalancing the arms enables the arms toremain in their current position when power is off; no torque needs tobe applied to any of the shoulder and arm joints to hold their currentpositions. With the arm balanced, the expenditure of electrical power isdedicated to the movement of payload, and motor current is used to senseand hold the tool, not the arm itself.

FIG. 4, FIG. 5, and FIG. 6 together illustrate the dimensions to whichone embodiment of the robotic manipulator can fold for purposes ofentering tight quarters. FIG. 4 shows a front view of the roboticmanipulator 10, FIG. 5 shows a top view of the robotic manipulator, andFIG. 6 shows a side view of the robotic manipulator. In a compact foldedposition, the video system 12 tilts downward, and the forearms 22 extendforward of the torso 16. The compact width 130, measured from theoutermost edges of the upper arms 20, is approximately 14 inches. Theheight 132, measured from the front edge of the back portion 16B of thetorso to the back edge of the back portion 16B, is approximately 5.2inches. The circle 134 (FIG. 4) illustrates an approximate size of aporthole into which the robotic manipulator 10 can project when broughtinto this compact, folded position. The diameter of such a porthole is14.125 in. The length 136 (FIG. 5), measured from the foremost edge ofthe forearms 22 to the rearmost edge of the back portion 16B of thetorso, is approximately 15 in. As other measures of its compactness, anembodiment of the robotic manipulator 10 can weigh as little as 5 kg andconsume as little as 50 W of electrical power.

FIG. 7 shows a diagram of an embodiment of the torso 16 of the roboticmanipulator 10 including the cylinder-shaped forward portion 16A and theback portion 16B. In the description of the torso 16, reference is alsomade to features described in connection with FIG. 3. Housed within theback portion 16B are a pair of motors 120-1, 120-2 (generally, 120).Each motor 120 is coupled by a gearbox 156 which turns a pulley 152(only the right pulley being shown), which is coupled by a belt 122(FIG. 3) to a pulley 164 (FIG. 8) on the shoulder 18. The forwardportion 16A includes a pair of axles 150 (only the right axle beingvisible), each axle 150 connecting to one of the shoulders 18. In thisembodiment, the motor 120-1 rotates the right shoulder 18-1; and themotor 120-2 rotates the left shoulder 18-2.

The torso 16 also has a printed circuit board (PCB) 154 (referred to asa motion control board) for controlling the motors 120. The PCB 154includes a servo controller, brushless motor commutation, PCM(pulse-code) modulation, closed-loop position/torque control, CANBUScommunications capability, DC supply, and an incremental positionencoder feedback. The PCB 154 can be implemented with a Maxon EPOS-2Module, produced by Maxon of Sachseln, Switzerland.

The torso 16 also includes an accelerometer (not shown) that senses adirection in which the host platform (e.g., a robotic arm, mobileplatform, or “jaws”) is holding the robotic manipulator 10 by sensing adirection of gravity. With knowledge of the direction of gravity and thedirection of the arms 14 (acquired from position encoders), the torso 16can manipulate the motors 120 to ensure the robotic manipulator remainscounterbalanced for the purposes of haptic feedback (i.e., torquesneeding to be applied to the joints in order for the robotic manipulatorto remain in position are not used in the determination of the hapticforces displayed in the hand controllers).

FIG. 8 shows a diagram of an embodiment of each right shoulder of therobotic manipulator 10. In the description of the shoulder, reference ismade to features described in connection with FIG. 3 and to the torso 16described in FIG. 7. Each shoulder 18 includes opposing shoulder blades160-1, 160-2 that extend orthogonally from a circular end-cap 162. Theend-cap 162 attaches to one side of the cylinder-shaped forward portion16A of the torso. Fixed to the end-cap 162 is a central pulley 164 witha bearing 166 for receiving one of the axles 150 of the torso. A belt122 fitted over the central pulley 164 couples the shoulder 18 to thepulley 152 of one of the motors 120 in the back portion 16B of thetorso. Operation of the motor 120 rotates the arm 14 coupled to theshoulder 18 about the axis 170-1. Associated with each motor 120 is aposition encoder so that the position of the motor (and, derivatively,the position of the shoulder) is known.

At the distal end of the shoulder blade 160-2 is a second pulley 172with a socket 174 for receiving a bearing 194 (FIG. 9) of one of theupper arms 20. A belt 116 (FIG. 3) fitted over the pulley 172 couplesthe shoulder 18 to the pulley of one of the motors 114 in the upper arm20. Operation of the motor 114 rotates the arm 14 coupled to shoulder 18about the axis 170-2. A third axis of rotation 170-3, described furtherin connection with FIG. 10, intersects the other two axes 170-1, 170-2at a point 176 within the shoulder 18.

Each shoulder 18 further includes a flex printed circuit board 178 thatwraps around the socket 166, extends along a length of the shoulderblade 160-1, and depends from the distal end of the shoulder blade160-1. This flex circuit 178 provides electrical power and digital datasignals to and from the motion control PCB contained in the upper arm,elbow and instrument subsystems. The coiled design permits rotation ofthe robot joints throughout their designed range of motion withnegligible resistance.

FIG. 9 shows a diagram of an embodiment of each upper arm 20 of therobotic manipulator 10. In the description of the upper arm 20,reference is made to features described in connection with FIG. 3 and tothe shoulder 18 described in FIG. 8. A major portion of the upper arm 20fits closely between the two shoulder blades 160-1, 160-2 of theshoulder 18. The upper arm 20 houses the motor 114 for driving the belt112 that engages the gear 202 of the elbow 24 (FIG. 10) and the motor114 for driving the belt 116 that engages a gear 202 (FIG. 10) of theelbow 24.

The upper arm 20 has a first opening 190 for receiving the gear 172 anda second opening 192 for receiving a shaft 200 (FIG. 10) of the elbow24. A rotatable shaft 194 at the opening 190 is adapted to enter closelythe bearing 174 of the gear 172, thereby coupling the upper arm 20 tothe shoulder 18. The shaft 194 coincides with the axis of rotation 170-2(FIG. 8).

The upper arm 20 further comprises electronics 196 for controlling theoperation of the motors 110, 114 (in response to commands). Each motor120 has an associated position encoder so that the position of the motor(and, derivatively, a corresponding position of the upper arm) is known.

FIG. 10 shows a diagram of an embodiment of the elbow 24 of the roboticmanipulator 10. In the description of the elbow 24, reference is made tofeatures described in connection with FIG. 3 and to the upper arm 20described in FIG. 9. The elbow 24 has a slight bend. At one end of theelbow 24 is a shaft 200 that enters the opening 192 (FIG. 9) in theupper arm 20 and a gear 202 that engages the belt 112. At the forked endof the elbow 24 is a forearm-mount 204 movably connected to a joint 206.Connected to this joint 206, the mount 204 can rotate about the axis208. A flex circuit 209 wraps around the joint 206, runs along aninterior surface of the forked end, and enters the body of the elbow 24.The flex circuit 209 is electrically connected to the flex circuit 178,and carries electrical power and digital communication signals to andfrom motor control PCBs 216 contained within the forearm main housing210 (FIG. 11) and accelerometer and angle position sensors contained inthe wrist 26 (FIG. 11).

FIG. 11 shows a diagram of an embodiment of the forearm 22 of therobotic manipulator 10. In the description of the forearm 22, referenceis made to features described in connection with FIG. 3 and to the elbow24 described in FIG. 10. The forearm 22 has a main housing 210 and atube-shaped portion 212 that extends from one end of the housing 210.The tube-shaped portion 212 ends at the wrist 26.

Within the housing 210 are four electromagnetic brushless motors withgear reductions 214, printed circuit boards 216, and a cable drivemechanism 218 that rotationally moves the tube shaped portion 212, thewrist 26 and gripper 28 in reaction to rotations of the electric motors214. The electric motors 214 may optionally be equipped withhigh-resolution rotational position sensors, or may be used without suchsensors by employing hall-effect sensors within the motor as coarseresolution rotational position sensors. The motors 214, PCBs 216, andcable drive system 218 are positioned within the housing 210 where theycan counterbalance the mass of the tube-shaped 212 and wrist 26. Thedesign of the forearm 22 is for its center of mass to be within thehousing 210 on the axis 220 passing through the forearm joint 222.

Mechanical features of the cable drive mechanism 218 separably connectto the forearm-mount 204 (FIG. 10) of the elbow 24. The forearm mount204 is connected to the both ends of the joint 206 of the elbow 24. Whenthe forearm 22 is attached to the elbow 24, the axes 208 (FIG. 10), 220align. When the forearm 22 moves, it rotates about the forearm joint222, which aligns with its center of mass; accordingly, the forearm 22remains balanced throughout its three-dimensional range of movement.

Attached to the wrist 26 are two gripper jaws 28. The jaws 28 can moveup and down across a plus or minus 90-degree range, and open and closefor cutting and grasping operations. The electronics 216 control themovement of the tool 28. A four-cable actuator (not shown), whichextends from the housing 210 of the forearm 22 through the tube-shapedportion 212 into the wrist 26, executes the yaw, pitch, open, and closemovements of the tool 28.

FIG. 12 shows a diagram of an embodiment of the wrist 26 of the roboticmanipulator 10. The wrist 26 is connected to the tool 28 at a rotatablewrist joint 230. The robotic manipulator 10 can accordingly roll itsfurthermost joint, the wrist joint 230 allowing the tool 28 to move plusor minus 90 degrees about the axis 232. The wrist 26 also includes aflex circuit 234 connected to both ends of the wrist joint 230 and to aprinted circuit board (PCB) 236. The flex circuit 234 carries digitalelectrical signals from two angular position sensors within the wristjoint 230. These signals encode data relating to the angular position ofeach of the gripper jaws 28.

The PCB 236 includes a surface-mounted 3-axis accelerometer 240 fordetecting vibration, thus providing vibrotactile feedback in threedimensions. The PCB 236 further contains a surface mount capacitancesensing circuit connected to flex circuit 234. These surface mountedIC's communicate digital electrical signals over I2C or similarinterfaces. The wrist 26 also includes wires (not shown) to the tip ofthe tool for sensing differences in electrical potential (voltage)between objects in contact with the grippers of the right and leftmanipulator, or the electrical current flowing through a wire being heldbetween the gripper jaws 28. A sensor (e.g., capacitive angular sensor230) senses and communicates the relative position of the tool 28 to thecomputational subsystem for processing into haptic control signals. Thisjoint-position sensing is independent of the motor position sensors. Inone embodiment, the PCB 236 communicates the data acquired by theaccelerometer 240 and the sensed joint position to the arm 14 over anI²C bus.

FIG. 13 shows a diagram of an embodiment of the stereoscopic videosystem 12 of the robotic manipulator 10, including the chain drive 106and the four rollers 102. The video system 12 includes two highdefinition (1080i or 720p) block camera modules 250-1, 250-2 (generally,250), with low light capability, auto focus, auto iris, a remote digitalinterface for configuration, zoom, and focus, and a component videooutput. In one embodiment, the field of vision for the camera modules250 is 50 degrees (wide angle) and 5.4 degrees (telephoto). These anglesmay be increased through the use of external wide angle adapters thatattach to the cameras 250.

Video subsystem 12 further includes an interface card (not shown) foraccess to data communication, a camera pitch (tilt) servo, a stereomicrophone preamplifier, and a video encoder (e.g., a Teredek Cube byTeredek of Irvine Calif) for performing low-latency video compression(e.g., H.264), stereo audio compression, and Ethernet transmission. Thecamera pitch servo pulls the video system 12 along the chain drive 106to tilt the video system 12 up and down. When the video system 12 istilted downwards through 90 degrees, rotating the robotic manipulator 10causes the binocular view to turn left and right (such rotation beingperformed by the host robotic arm to which the robotic manipulator 10 isattached). The view direction of the camera affects the direction ofmovement of the robotic arms and hands (i.e., tools). For example, ifthe camera direction is tilted downwards, the arms move downward inresponse to a reaching-forward movement of the hand controls by theoperator at the remote control station—there is a remapping of theoperator's coordinate system to the coordinate system of the binocularview.

Reference in the specification to “one embodiment” or “an embodiment”means that a particular, feature, structure or characteristic describedin connection with the embodiment is included in at least one embodimentof the teaching. References to a particular embodiment within thespecification do not all necessarily refer to the same embodiment. Theterminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method, and computer programproduct. Thus, aspects of the present invention may be embodied entirelyin hardware, entirely in software (including, but not limited to,firmware, program code, resident software, microcode), or in acombination of hardware and software. All such embodiments may generallybe referred to herein as a circuit, a module, or a system. In addition,aspects of the present invention may be in the form of a computerprogram product embodied in one or more computer readable media havingcomputer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wired, optical fiber cable, radio frequency (RF), etc. or any suitablecombination thereof.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as JAVA, Smalltalk, C++, and Visual C++ or the like andconventional procedural programming languages, such as the C and Pascalprogramming languages or similar programming languages.

Aspects of the present invention may be described with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

Aspects of the described invention may be implemented in one or moreintegrated circuit (IC) chips manufactured withsemiconductor-fabrication processes. The maker of the IC chips candistribute them in raw wafer form (on a single wafer with multipleunpackaged chips), as bare die, or in packaged form. When in packagedform, the IC chip is mounted in a single chip package, for example, aplastic carrier with leads affixed to a motherboard or other higherlevel carrier, or in a multichip package, for example, a ceramic carrierhaving surface and/or buried interconnections. The IC chip is thenintegrated with other chips, discrete circuit elements, and/or othersignal processing devices as part of either an intermediate product,such as a motherboard, or of an end product. The end product can be anyproduct that includes IC chips, ranging from electronic gaming systemsand other low-end applications to advanced computer products having adisplay, an input device, and a central processor.

While the invention has been shown and described with reference tospecific preferred embodiments, it should be understood by those skilledin the art that various changes in form and detail may be made thereinwithout departing from the spirit and scope of the invention as definedby the following claims.

What is claimed is:
 1. A robotic manipulator comprising: a body; astereoscopic video system movably coupled to the body, the stereoscopicvision system producing a stereoscopic video of an environment of therobotic manipulator; and two independently remotely controlled armscoupled to opposite sides of the body, each arm moving in proprioceptivealignment with the stereoscopic video produced by the stereoscopic videosystem in response to commands received from a remote control stationbased on movements performed by an operator at the remote controlstation.
 2. The robotic manipulator of claim 1, wherein the body isadapted to mount to a mobile host platform.
 3. The robotic manipulatorof claim 1, wherein the body further comprises an accelerometer fordetecting a direction of gravity, and the robotic manipulator furthercomprises a computational subsystem that balances the arms of therobotic manipulator in response to the detected direction of gravity andposition of the arms.
 4. The robotic manipulator of claim 1, wherein thebody has rails and the video system has an undercarriage with rollersthat ride on the rails of the body, and the robotic manipulator furthercomprises a chain drive coupling the video system to the body and aservo that moves the video system along the chain drive to tilt thevideo system up and down.
 5. The robotic manipulator of claim 1, whereineach arm comprises an upper arm rotatably coupled to one side of thebody for rotational movement about first and second axes, an elbowrotatably joined to the upper arm for rotational movement about a thirdaxis, the three axes of rotation intersecting at a point within theupper arm.
 6. The robotic manipulator of claim 5, wherein each armfurther comprises a forearm rotatably joined to the elbow of that armfor rotational movement about an elbow joint, the forearm having acenter of mass at the elbow joint to balance the forearm at the elbowjoint.
 7. The robotic manipulator of claim 6, wherein each forearmcomprises a wrist adapted to hold an instrument.
 8. The roboticmanipulator of claim 7, wherein each upper arm comprises a first servothat rotates the upper arm about one of the first and second axes and asecond servo that rotates the upper arm about the other of the first andsecond axes, the servos being disposed within the upper arm at locationswhere the servos provide a counterbalance to a remainder of the arm. 9.The robotic manipulator of claim 1, wherein each arm comprises arotatable wrist with a tool, the wrist having a three-axis accelerometerthat detects and reports vibrations in three dimensions.
 10. The roboticmanipulator of claim 9, wherein the wrist further comprises means fordetecting a degree of grip opening by the tool.
 11. A system comprising:a robotic manipulator comprising: a servo actuator subsystem includingtwo independently remotely controlled arms coupled to opposite sides ofa body; a video-capture subsystem that produces a stereoscopic video ofa local environment of the robotic manipulator; and a computational hostsubsystem transmitting movement commands to the servo actuator subsystemthat cause the servo actuator subsystem to move each arm inproprioceptive alignment with the stereoscopic video produced by thevideo-capture subsystem.
 12. The system of claim 11, further comprisinga remote control station having haptic-interface devices, the remotecontrol station sending movement commands to the computational hostsubsystem of the robotic manipulator over a network in response tomanipulation of the haptic-interface devices by an operator.
 13. Thesystem of claim 12, wherein the computational host subsystem transmitsfeedback signals received from the servo actuator subsystem and videodata received from the video-capture subsystem to the remote controlstation.
 14. The system of claim 13, wherein at least some of thefeedback signals correspond to vibrations detected in three dimensionsby wrists of the arms.
 15. The system of claim 13, wherein at least someof the feedback signals correspond to a sense of grip effort beingexerted by a tool held by a wrist of one of the arms.
 16. The system ofclaim 11, wherein the computational host subsystem balances the arms ofthe robotic manipulator in response to a detected direction of gravityand position of the arms.
 17. The system of claim 11, further comprisinga mobile host platform to which the robotic manipulator is mounted. 18.The system of claim 17, wherein the robotic manipulator furthercomprises a power distribution subsystem that distributes power acquiredfrom the mobile host platform to the other subsystems of the roboticmanipulator.
 19. The system of claim 11, further comprising a mobilehost platform having a manipulator arm with a gripper at a distal end ofthe manipulator arm, and wherein the robotic manipulator is in thegripper at the distal end of said manipulator arm.
 20. The system ofclaim 11, wherein each arm of the robotic manipulator comprises an upperarm rotatably coupled to one side of the body for rotational movementabout first and second axes, an elbow rotatably joined to the upper armfor rotational movement about a third axis, the three axes of rotationsubstantially intersecting at a joint within the upper arm.
 21. Thesystem of claim 20, wherein each upper arm comprises a first servo thatrotates the upper arm about one of the first and second axes and asecond servo that rotates the upper arm about the other of the first andsecond axes, the servos being disposed within the upper arm at locationswhere the servos provide a counterbalance to a remainder of the arm. 22.The system of claim 20, wherein each arm further comprises a forearmrotatably joined to the elbow of that arm for rotational movement aboutan elbow joint, the forearm having a center of mass at the elbow jointto balance the forearm at the elbow joint.
 23. The system of claim 22,wherein each forearm comprises a wrist adapted to hold a tool.
 24. Amethod of remotely operating a robotic manipulator having a body, astereoscopic video system movably coupled to the body, and two remotelycontrolled arms coupled to opposite sides of the body, the methodcomprising: capturing, by the video system, a stereoscopic video of anenvironment of the robotic manipulator; independently moving, inresponse to commands received from a remote control station, each arm inproprioceptive alignment with the stereoscopic video produced by thestereoscopic video system.
 25. The method of claim 24, furthercomprising transmitting feedback signals received from the arms and thestereoscopic video from the video system to the remote control station.26. The method of claim 25, further comprising presenting, on a displayscreen at the remote control station, a display overlay that shows acommanded configuration of the arms in order to compensate for latencyof the stereoscopic video.
 27. The method of claim 25, furthercomprising delaying generation of a haptic response associated with thefeedback signals at the remote control station in order to synchronizethe haptic response with display of the stereoscopic video.
 28. Themethod of claim 24, further comprising sensing a direction of gravity inthe body and a position of the arms, and balancing the roboticmanipulator in response to the sensed direction of gravity and armpositions.
 29. The method of claim 24, further comprising mounting therobotic manipulator to a mobile host platform.
 30. The method of claim24, further comprising mounting the robotic manipulator to a distal endof a manipulator arm attached to a mobile host platform.